FROST—Fast row-stochastic optimization with uncoordinated step-sizes
نویسندگان
چکیده
منابع مشابه
Non-parametric Stochastic Approximation with Large Step sizes
HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau...
متن کاملOnline Learning with Adaptive Local Step Sizes
Almeida et al. have recently proposed online algorithms for local step size adaptation in nonlinear systems trained by gradient descent. Here we develop an alternative to their approach by extending Sutton’s work on linear systems to the general, nonlinear case. The resulting algorithms are computationally little more expensive than other acceleration techniques, do not assume statistical indep...
متن کاملGradient Methods with Adaptive Step-Sizes
Motivated by the superlinear behavior of the Barzilai-Borwein (BB) method for two-dimensional quadratics, we propose two gradient methods which adaptively choose a small step-size or a large step-size at each iteration. The small step-size is primarily used to induce a favorable descent direction for the next iteration, while the large step-size is primarily used to produce a sufficient reducti...
متن کاملAlmost sure convergence of stochastic gradient processes with matrix step sizes
We consider a stochastic gradient process, which is a special case of stochastic approximation process, where the positive real step size an is replaced by a random matrix An: Xn+1 = Xn Anrg(Xn) AnVn: We give two theorems of almost sure convergence in the case where the equation rg = 0 has a set of solutions. Key Words and Phrases: Stochastic approximation, stochastic gradient AMS 1991 Subject ...
متن کاملAsymptotic Properties of Two Time-Scale Stochastic Approximation Algorithms with Constant Step Sizes
Asymptotic properties of two time-scale stochastic approximation algorithms with constant step sizes are analyzed in this paper. The analysis is carried out for the algorithms with additive noise, as well as for the algorithms with non-additive noise. The algorithms with additive noise are considered for the case where the noise is state-dependent and admits the decomposition as a sum of a mart...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: EURASIP Journal on Advances in Signal Processing
سال: 2019
ISSN: 1687-6180
DOI: 10.1186/s13634-018-0596-y